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ABSTRACT 

Medical expertise research methods were used to 
explore the relationship between auditing expertise and case 
representation. Subjects were 8 first-year economy students, 8 
fourth-year auditing students, 8 postgraduate students in auditing, 
and 8 experienced auditors in the Netherlands, ranging in experience 
from only a limited knowledge of bookkeeping to experience of over 12 
years. Subjects were presented with three different cases that they 
diagn».sed and recalled. Results show that the level of expertise has 
a significant effect on the accuracy of subjects' diagnoses and 
inferred recall, although subjects did not differ significantly in 
lit eral recall. It is concluded that hypotheses generalized from 
medical expertise research are to a great extent applicable to the 
auditing domain. It may also be that subjects process descriptive and 
numerical information in different ways. Three figures illustrate the 
discussion, (Contains 11 references.) (SLD) 
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Abstract 

In the present study medical expertise research methods are used to explore the relationship between 
auditing expertise level and case representation. The subjects were presented three different cases, 
which they diagnosed and recalled. The results showed that level of expertise did have a significant 
effect on accurateness of diagnosis and inferred recall, yet subjects did not differ significantly in 
literal recall. 



Inii^uction 

Auditing expertise studies emphasizing knowledge structures have started in 
the late 1980's (Libby, 1985; Choo, 1989). Presently, an adequate theoretical basis is 
still missing. Since research on knowledge structures in the domains of physics and 
the medicine have started earlier and probably progressed more, auditing expertise 
research might benefit from research methods developed in these domains. For two 
reasons the medical domain might be important for the auditing field. First, the 
educational program of both physicians and auditors consists of a theoretical and a 
practical part. Second, according to Ashton, Kleinmuntz, Sullivan & Tomassini, 
(1988) and Boshuizen & Schmidt (1992), at a generic level the task of a physician and 
an auditor is essentially the same. Like a physician, an auditor diagnoses and if 
necessary provides treatment. The diagnostic task we are referring to is the audit of 
financial statements. Auditors use different types of key controls to arrive at several 
partial diagnoses before they decide whether a financial statement is in accordance 
with established criteria. Key controls are for example the separation of duties, the 
existence of an approved price Ust or the presence of possible errors in the financial 
statements. 

Since the same generic task may require the same kind of knowledge 
structures (Boshuizen & Schmidt, 1992) and research on medical expertise has 
further developed than auditing expertise research, we took expertise research from 
the medical domain as a theoretical basis for this study. 

In the medical domain two dependent variables have been described which 
show a clear relationship with expertise level: quaUty of diagnosis and amount of 
recall. The relationship between expertise level and quality of diagnosis was 
examined by Boshuizen (1989), Boshuizen and Schmidt (1992) and Feltovich, 
Johnson, Moller and Swanson (1984). All researchers found an increasing accuracy 



ERIC 



3 



3 



of diagnosis with increasing expertise. Recall studies assume that recall reflects 
aspects of the internal representation of a problem, a more coherent and extensive 
representation would result in more and better recall. This hypothesis was 
supported by Norman, Brooks and Allen (1989). The observation was however 
complicated by other research outcomes in which an inverted U-shaped relation 
was found between expertise level and recall (Muzzin, Norman, Feighner & 
Tugwell, 1983; Patel & Medley-Mark, 1985; Schmidt & Boshuizen, 1993). Contrary to 
the other studies, Norman et al. (1989) used numerical stimuli. These numerical 
stimuli might have forced subjects into an analytic processing mode computing and 
comparing data and deducting conclusions from them. The verbal case descriptions 
used by Muzzin et al (1983), Patel and Medley-Mark (1985) and Schmidt and 
Boshuizen (1993) might have called for direct schema activation and instantation, 
not requiring any analytic activity. This difference in processing mode might have 
caused the different shapes of the relations between expertise and recall found in the 
studies. Since auditing is predominantly a numerical domain, it might be expected 
that the linear results by Norman et al. (1989) could be generalized to the auditing 
domain. 

In addition there appears to be a distinction between literal recall and 
inferred recall, Boshuizen and Schmidt (1992) and Coughlin and Patel (1986) found 
an increase of inferences with increasing expertise. 

In summary, studies in auditing are expected to display results similar to 
those found in medicine. Accuracy of diagnosis, Hteral recall and inferences may 
vary linearly with expertise level. 

Method 

Subjects. Eight first-year economy students, eight fourth-year auditing students, 
eight post-graduate students and eight experienced auditors participated in this 
study. The first-year students had only limited knowledge of bookkeeping. The 
fourth-year students almost had their masters degree, they did not have any 
practical experience in auditing. The post-graduate subjects worked as in-charge 
auditors and were part-time students in a post-graduate training program at the 
University of Limburg. Their practical experience equalled 5 years on the average. 
The fourth group consisted of certified public accountants, who were auditors with 
an average experience of 12.5 years. These experts worked in different auditing 
firms. 
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Material The stimulus material consisted of a description of the internal control 
structure, a balance sheet and a profit-and-loss account on paper, which is a 
summary the information an auditor gets in real life. There were three different 
types of business organizations: a Wholesale company, a Contractor firm and a 
Foundation. Case length was one page. 

Procedure . Subjects were told that three cases would be presented and that it would 
be their task to write down their diagnoses and recall. They studied the cases as long 
as they wished. After that it was not allowed to look at the cases any Jonge^ Total 
duration was one and a half hour for almost every subject. Case presentation order 
was balanced over subjects. 

Analysis . Subjects* diagnoses were compared with a list of eleven diagnoses which 
considered internal control information, balance sheet and profit-and-loss account 
figures. This list of diagnoses was taken as a standard list. Examples of these 
diagnosis are seperation of functions, various warehouses, finance ratios, money 
lend,. high interest. For each case, the maximum diagnostic score was eleven. 
Free recall protocols were divided in small meaningful information units referred to 
as propositions. These propositions were matched to the case propositions. Amount 
of literal recall was determined by counting the number of propositions similar to 
case propositions. These raw scores were corrected for differences in case length by 
dividing it by the total number of case propositions. 

Inferred recall consisted of the number of correct conclusions and interpretations 
based on two or more case propositions. 

All data were analysed by means of a four (levels of expertise) by three (cases) by 
three (presentation order) analyses of variance with repeated measures. 

Results and Discussion 

Diagnostic i^ccuracv On the next page, figure 1 shows the average number of 
accurate diagnoses proposed by the subjects. As predicted the experts produced a 
more accurate diagnosis than the other subjects, F(3,28) = 46.12, u < -OOOl- Yet, the 
figure shows that the expected linearity is not at hand. Although each standard 
diagnosis contained the same number of elements, fourth-year, post-graduate and 
experts mentioned most diagnoses in the Foundation case. There was a significant 
case effect, F(2,26) = 5.19, p < .009. Probably, subjects diagnosed the Foundation case 
better than the other cases, because of the relatively clearer distinction between 
cycles (sales, purchase, inventory, debt, etc.). 
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0 1 1 1 1 ' 

1 st-year 4th-year post-grad, experts 

Figure 1. Average diagnostic accuracy as a function of level of expertise 

Amount of Literal Recall In figure 2 the average percentage of recall is presented. 
No significant effect of expertise was found, F(3,26) = .851, u < -479. Cases did have a 
significant effect on literal recall, F(2,26) = 9.52, p < .0003. The Contractor case and 
the Wholesale case showed some differences between the subjects, the Foundation 
case did not result in different recalls. 
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1 st-year 4th-year postgrad. experts 

Figure 2 Average percentage of literal recall as a function of level of expertise 
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Inferences in Recall Figure 3 depicts the average amount of inferences in recall. 
Expertise had a significant effect on the number of inferences, F(3,26) = 8.76, 
E < .0003. This result supported the hypothesis that the more experienced an 
auditor, the more inferences he makes. The cases also had a significant effect, 
F(2,26) = .4.24, £ < .0198. The Contractor case leads to more inferences overall. 
According to the expert subjects this case was considerably more complex than the 
other ones. Data on diagnosis in Figure 1 support this observation. In general, this 
might indicate that subjects make more inferences in the recall as the presented 
information demands more processing capacity. 




1st-year 4th-year post-grad, experts 
Figure 3 Average inferences in recall as a function of expertise. 



In summary, it can be concluded that the hypotheses generalized from 
medical expertise research are to a great extent applicable to the auditing domain. 
Expertise had a significant effect on diagnostic accuracy and number of inferences 
in recall. Yet, expertise did not result in a significant effect on recall. The 
explanation for not finding this effect may have been the type of stimulus material 
we used. The material consisted of two different types of information, descriptive 
and numerical information. It is possible that subjects process these types of 
information in different ways. Further qualitative analyses will be discussed during 
the presentation of this paper at the AERA meeting. 



7 



References 

Ashton, R.H., Kleinmuntz, D.N., Sullivan, J.B. & Thomassini, L.A. (1988). Audit 

decision making. In Abdel-Khalik, A.R. & Solomon, I. (Eds.). Research 

opportunities in auditing: the second decade. 
Boshuizen, H.P.A. (1989). De ontwikkeling van medische expertise; een cogrutief 

psychologisch benadering. [ On the development of medical expertise; a 

cognitive psychological approach]. Doctoral dissertation. Haarlem, The 

Netherlands: Thesis Pubhshers. 
Boshuizen, H.P.A. & Schmidt, H.G. (1992). The role of biomedical knowledge in 

clinical reasoning by experts, intermediates and novices. Cognitive science, 

16, 153-185. 

Choo, F. (1989). Cognitive scripts in auditing and accounting behavior. Accounting, 

Organizations and Society, 14, 481-493. 
Coughlin, L.D. & Patel, V.L. (1986). Text comprehension and expertise in the 

dorr.ain of medicine. Paper presented at the Annual Meeting of the American 

Educational Research Association. San Fransisco, CA. 
Feltovich, P.J., Johnson, P.E., Moller, J.H. & Swanson, D.B. (1984). LCS: The role 

and development of medical knovs^ledge in diagnostic expertise. In: W.J. 

Clancey & E.H. ShortUffe (Eds.). Readings in medical artificial intelligence; 

the first decade. Cambridge MA: Addison-Wesley. 
Libby, R. (1985). Avialability and the generation of hypotheses in analytical review. 

Journal of Accounting research, 23, 648-667. 
Muzzin, L.J., Norman, G.R., Feightner, J.W. & Tugwell, P. (1983). Expertise in 

recall of clinical protocols in two specialty areas. Proceedings of the 22nd 

Conference on Research in Medical Education. Washington, DC. 
Norman, G.R., Brooks, L.R. & Allen, S.W. (1989). Recall by expert medical 

practitioners and novices as a record of processing attention. Journal of 

experimental psychology: Learning, Memory and Cognition,15, 6, 1166-1174. 
Patel, V.L. & Medley-Mark, V. (1985). Knowledge, integration from clinical texts, 

use of factual, inferential and integrative questions. Proceedings of the 24th 

Annual Conference of Research in Medical Education. Washington, DC. 
Schmidt, H.G. & Boshuizen, H.P.A. (in press). On the origin of intermediate effects 

in clinical case recall. Memory & Cognition. 



8 



